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Abstract 

We study the spin-^ Heisenberg antiferromagnet with an antiferromagnetic J3 
(third nearest neighbor) interaction on a square lattice. We numerically diago- 
nalize this "Ji- J3" model on clusters up to 32-sites and search for novel ground 
state properties as the frustration parameter J3/J1 changes. For "larger" 
J3/J1 we find enhancement of incommensurate spin order, in agreement with 
spin-wave, large-TV expansions, and other predictions. But for intermediate 
J3/ 1 Jii the low lying excitation energy spectrum suggests that this incommen- 
surate order is short-range. In the same region, the first excited state has the 
symmetries of the columnar dimer (spin-Peierls) state. The columnar dimer 
order parameter suggests the presence of long-range columnar dimer order. 
Hence, this spin-Peierls state is the best candidate for the ground state of the 
J1-J3 model in an intermediate J3/J1 region. 
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The phase diagram of the frustrated spin-| Heisenberg antiferromagnet has received 
much interest in recent years. On a square lattice, frustration can be introduced by further- 
than-nearest neighbor antiferromagnetic couplings. Short-range interactions up to a distance 
of two lattice constants have been studied. This is the "Ji- J2-J3" model, which is described 
by the Hamiltonian 

h = Ji^Si ■ Sj + j 2 ^2Si ■ Sj + j 3 ^2 • Sj, (i) 

nn 2nn 3nn 

where the sums run over all first, second, and third nearest neighbors, and all Jj > 0. 
The classical phase diagram of this model is well-known to have transition lines between 
Neel, collinear, and spiral states The critical line separating the Neel and spiral states, 
Ji — 2J 2 — 4J 3 = 0, is called the classical critical line (CCL). The quantum phase diagram 
is less clear. When the frustration is small (at small Ji and J3), the model possesses Neel 
order. Various analytical studies including linear spin-wave 0|| and mean-field theories || 
have shown that the ground state possesses collinear and spiral (incommensurate) spin order 
at large J2 and J3 respectively. The ground state at intermediate J 2 and J3, particularly 
along the quantum analog of the CCL, is still controversial. While some theories [Q-fl predict 
that frustration and quantum fluctuation destroy the Neel order to form a state without spin 
order, others |5||| predict that quantum fluctuation can stabilize the Neel state along this 
critical line. Nevertheless, it seems likely that in between the Neel and spiral phases, there 
exists an intermediate state without spin order. Large-iV expansions of the unfrustrated 
antiferromagnet |7j] predict that this intermediate state is spontaneously dimerized. This is 
supported by series expansion on the J1-J2-J3 model JT|, although the former (Ref. 0) did 
not treat the frustration due to J2 and J3 explicitly. On the other hand, spin-wave theory 
0110 predicts that this intermediate state is a spin-liquid. 

The search for a spin-liquid state in low- dimensional quantum antiferromagnets has long 
been a fascinating problem. Such a state is most likely to be found in frustrated systems 
with large quantum fluctuations. Therefore, the region along the quantum analog of the 
CCL in the J1-J2-J3 model is a good place to search for a spin-liquid state, although other 
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kinds of long-range order (such as spin-Peierls order) have been proposed in the same region. 
Since the location of this critical line is unknown, it is tedious to work with two adjustable 
parameters J 2 and J3. Most numerical diagonalization studies of the frustrated Heisenberg 
model have been for J 3 = |||| . A recent study || has suggested that the end of the critical 
line on the J2 axis is a Lifshitz point and thus not representative of the whole critical line. 
The purpose of this letter is to study this model on the J3 axis. In the following, we will 
take J\ — 1 and J2 = 0. 

Using the Lanczos algorithm, we are able to diagonalize this J1-J3 model on a 32-site 
square cluster. Most results in this letter are obtained from the 16-site (4x4) and 32-site 
square lattices. It is obvious that the 16-site lattice is too small to include the J 3 interaction 
because each site has only two (instead of four) distinct third nearest neighbors. Hence we 
will use results obtained from the 16-site system for comparison only. Finite-size scaling of 
the 16- and 32-site results will not be reliable, except perhaps at small J3. 

It is well known that for J 3 = 0, the ground state of the J1-J3 model exhibits long-range 
Neel order [|10|| . Fig. [I] shows the finite-size plot of the staggered magnetization , defined 



as m 



S(tt 1 tt)/N (see Eq. @). The system sizes are N = 16, 24 |TJ, and 32. The 1/y/N 



dependence is taken from spin- wave theory for the unfrustrated case [|EJ. We can see that 
at J 3 = 0.3, extrapolates to a finite value as iV — > 00. But the linear extrapolation fails 
at J 3 = 0.35. Hence we conclude that the Neel order persists at least up to J 3 = 0.3 in 
the thermodynamic limit. To study the spin order as J3 increases further, we calculate the 
static structure factor, 

s(q) = ^£^- R <)<s fe .s z >. (2) 

Fig. |2| shows 5(q) for the 32-site square lattice at different J3. It clearly shows that as J3 
increases from zero, the peak shifts from (ir,n) to (^f, ^) at J 3 ~ 0.5, and then to (~, |) 
at J 3 ~ 0.7. This shows that the Neel order vanishes as J3 increases and another spin 
order develops which has ordering vector along the (1,1) direction. If the system possesses 
incommensurate spin order, the peak in S(q) should shift continuously from (tt,tv) to (|, |) 
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as J3 increases. Due to the discrete nature of the cluster, such a continuous shift along the 
(1, 1) direction is not possible. Nevertheless, Fig. |2| does indicate that incommensurate spin 
order develops as J3 increases. We also calculate the dynamic structure factor ||, S(q,uj). 
Sharp low energy peaks are found at momenta along the (1, 1) direction. As J3 increases, 
the lowest energy peak changes from (tt, 7r) to (2£ ^f) at J 3 ~ 0.5, and then to (|, |) at 
J3 ~ 0.7, indicating that the Neel order vanishes and the system develops another spin order 
which has ordering vector along the (1, 1) direction. 

To study whether this incommensurate spin order is long-range, we calculate the twist 



correlation function [|l3|j , 



Xt 



^ r X (Sr+x + S r+y/ 



r 



(3) 



where x and y are unit vectors. We expect Xt to be independent of N for large enough 
N if the system possesses long-range incommensurate spin order. Fig. |3|(a) shows Xt m 
the 16- and 32-site systems at different J 3 . In both systems, Xt is enhanced at J3 larger 
than about 0.4. This enhancement suggests the existence of incommensurate spin order at 
large J3, which is consistent with the findings from the static and dynamic structure factors 
discussed above. However, only spin- wave excitations will show up as peaks in >S(q, u). 
Therefore, it does not exclude the existence of singlet excitations, especially at intermediate 
J3. In particular it is difficult to judge from Fig. |3|(a) whether the incommensurate order is 
long-range at intermediate J3. 

If a system possesses a broken symmetry in the thermodynamic limit, the ground state 
of the finite system will still be fully symmetric. In this case the ground state expectation 
of the appropriate order parameter will have long-range correlations, and there will exist 
low lying excited states with the appropriate symmetries whose energy gaps vanish in the 
thermodynamic limit |TJJ. Consequently, we can use the low lying energy levels of a finite 



system to study the possible existence of long-range order. Fig. |] shows the energies of a 
few low lying eigenstates in the 32-site system. For J3 smaller than about 0.4, the first 
excited state is a triplet with momentum (vr, 7r), consistent with the existence of Neel order 



for small J3. We denote this state as Eti- For J3 larger than about 0.85, the first excited 
state is a triplet with momentum (|, |). We denote this state as Et2- Eti and Et2 are 
the two spin- wave excitations which show up as low energy peaks in S(q,u). However, 
at intermediate J3, states with momentum (^, ^r) are never the first excited state. We 
denote the first excited state in this region as Eg- It is a two-fold degenerate singlet pair, 
one with momentum (0, 7r) and the other with (ir, 0). Both are odd under reflection along 
the direction orthogonal to their momenta. The symmetries of the Eg state resemble the 
columnar dimer state fl5[]. In this state, nearest neighbor spin pairs form singlets (dimers), 



and these dimers freeze into a columnar order. It is four- fold degenerate, and can form four 
states with distinct symmetries: two with zero momentum, of which one is fully symmetric 
while the other is odd under rotation; and two with momenta (0, 7r) and (it, 0). The last two 
have the same symmetries as the degenerate Eg state. 

Since the finite system always has a first excited state no matter whether the ground 
state possesses true long-range order, the results of the above study of the low lying states 
alone are not sufficient to show the existence of dimer order. The next evidence comes from 



the order parameter for the columnar dimer state |16 | 



■irS r -S r+x + i(-lpS r -S r+y . (4) 



In finite-size calculations, we examine the correlation function 

1 



Xdv 



Endirn 
, 9 * 
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(5) 



If the ground state has long-range columnar dimer order, Xdim ~ 0(1). Fig. |3](b) shows Xdim 
at various J3 in the 16- and 32-site systems. Both systems have a peak in Xdim, indicating 
that columnar dimer order is enhanced in the corresponding region of J3. In the 32-site 
system, the peak is at J 3 ~ 0.7, which corresponds to the minimum energy gap between E 
and Eg in Fig. |j. The different peak positions in Fig. |3|(b) may be due to finite-size effect 
of the 16-site lattice as discussed above. This effect also prevents us from doing a reliable 
finite-size scaling study of the peak values of Xdim- 



The dimer correlations can be demonstrated clearly by calculating the dimer-dimer cor- 



relation function |T7| defined as 

C(ij)(k,i) = ((Si ■ Sj)(S k ■ Si)} - (Si ■ Sj) 2 , (6) 

where the bracket (m, n) denotes nearest neighbor sites. A dimer liquid state will display 
short-range structure in C^j^t) but decrease to zero at large dimer separations. On the 
other hand, a dimer solid, or spin-Peierls state, will continue to show periodic oscillations 
reflecting the underlying long-range order. C(ij\n.,i) f° r & U inequivalent dimer pairs of the 32- 
site lattice evaluated at J3 = 0.7 are tabulated in Table |. Fig. |^ is a pictorial representation 
of C(ij)(kj). The reference bond is represented by a double line. For all other bonds 
(k,l), the magnitude of C(ij)(k,j) is represented by the thickness of the line joining sites 
k and I. Solid lines represent positive correlation, and broken lines represent negative or 
anti-correlation. It is clear that nearest neighbor spin pairs tend to form dimers, and the 
dimers are arranged in a columnar fashion. The dimer-dimer correlations do not decrease 
appreciably in the largest dimer separation allowed in our system size. 

To conclude, our numerical results show that Neel order in the J1-J3 model is stable up 
to J3 > 0.3, as compared to J3 = 0.25 in the classical case. This could be the result of the 
CCL being moved to much larger J 2 and J3 values by quantum fluctuation |J, when the Neel 
state is stabilized (order from disorder) and the spiral state is destabilized along the critical 
line. Our results further show that the model is likely to have a spin-Peierls state between 
the Neel state (at small J3) and the incommensurate state (at large J3), in agreement with 
theoretical predictions [p^fFUPSl . In particular, Ref. 0] predicted that the dimerized patterns 
depend on the spin S, and our results agree with it for S — 1/2. We would like to remark 
that Ref. [JF] did not treat the frustration explicitly, and it is not trivial that it gives the right 
prediction along the quantum analog of the CCL. But when the same analysis is extended to 
include frustrations due to J 2 and J3, similar dimerization patterns are found [|B|. At large 
J3, we believe, as our results show, that the model possesses incommensurate spin order. 
But our finite cluster has no wave vector from (^, ^) to (|, |) along the (1, 1) direction. 
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Hence we are not able to locate the transition point to the incommensurate state nor to 
study the order of the transition. 
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FIGURES 

FIG. 1. Finite-size plot of the staggered magnetization m) at different J3. The straight lines 
are the best fit to the data. The dotted lines are straight lines joining the data points. 

FIG. 2. The static structure factor for the 32-site lattice at different values of J3. 

FIG. 3. (a) xt and (b) Xdim at different J3 in the 16-site (•) and 32-site (+) systems. 

FIG. 4. Lowest energy states in different momentum sectors at different J3 in the 32-site system. 
Eq is the ground state. Eti and Eti are spin triplets while Es is a spin singlet. For the purpose 
of clarity, lowest energy states in other momentum sectors and the second lowest energy state with 
momentum q = (0, 0) are not shown. These states have higher energy than the second lowest 
energy states shown in the figure. 

FIG. 5. Dimer-dimer correlation function C^ t2 3)(k,l) °f the 32-site system at J3 = 0.7. The 
reference bond (19, 23) is represented by a double line. The magnitude of Cn 9t 23)(k,l) is proportional 
to the thickness of the line joining the pair of sites (A;, The solid line means C( 19i2 3)(fc,«) is positive, 
and the broken line means C^^z){k,l) is negative. 
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TABLES 

TABLE I. Dimer-dimer correlation functions for all inequivalent dimer pairs in the 32-site 
system at J3 = 0.7. The reference pair is (19, 23). See Fig. 5 for the numbering of the sites. 



(k,l) 


C(19,23)(fe,/) 


(k,l) 


C(19,23)(fc,0 


(L5) 


0.055075 


(18,22) 


-0.054987 


(1,29) 


0.003783 


(19,22) 


-0.019733 


(2,29) 


0.051208 


(21,25) 


0.003865 


(5,10) 


-0.051912 


(21,26) 


-0.045224 


(10,14) 


0.067346 


(22,26) 


0.009617 


(13,18) 


0.051260 


(22,27) 


0.104930 


(14,19) 


-0.071722 


(25,29) 


-0.042596 


(17,21) 


0.050038 


(26,29) 


0.003208 


(18,21) 


0.002528 


(26,30) 


0.067526 
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Leung and Lam Fig. 1 




Leung and Lam Fig. 2 
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